The present invention relates to tracking cameras that use a dual-view optical system to permit the generation of a first image with a narrow-angle field of view and a second image with a wide-angle field of view. The narrow-angle image is used to produce a high quality output image, and the wide-angle image is used to improve tracking performance of the tracking system. Both images may be maintained in continuous logical registration with respect to one another so that the tracking system can determine how the object in one image should appear in the other image.
A camera tracking system is one that automatically maintains a subject in view of the camera. This can be accomplished by acquiring a target and maintaining the target in the central view area of the tracking camera. Various kinds of tracking systems are known. Maintenance of the target in the view of the camera by the tracking system can be accomplished based on the definition of the tracked object using some kind of artificial intelligence mechanism, manually, through voice commands, or by a combination of these.
Conventional tracking systems typically track moving objects using a camera with an image detector to capture images of the object. These captured images are then processed to find and track the object. When a camera with a relatively narrow field of view is used to capture the images, and the object being tracked (OBT) moves far away from the center of the camera""s field of view, the camera""s aim must be adjusted to continue the tracking process. For example, if the OBT moves to the right, the camera""s aim would be adjusted to the right.
With conventional tracking systems, it is best to adjust the camera""s aim before the OBT moves completely out of the camera""s field of view, because once the image of the OBT is lost, the object must be reacquired. With fast-moving objects, however, it can be difficult for the tracking system to adjust the camera""s aim before the OBT xe2x80x9cfalls offxe2x80x9d the edge of the image. This condition where the OBT falls off the edge of the image is referred to herein as a xe2x80x9ctracking dropoutxe2x80x9d.
When a tracking dropout occurs in a conventional tracking system, the system can attempt to reacquire the OBT by panning and tilting the camera along a predetermined search pattern. But reacquiring the OBT can be difficult or impossible, particularly in cases where the OBT continues to move about rapidly. In the meantime, until the OBT is reacquired, the tracking system remains unaware of the OBT""s location. While tracking dropouts may be acceptable in certain applications, they are not acceptable in other applications such as security systems and camera systems that track balls at sporting events.
Using wide-angle focusing optics for tracking can help avoid tracking dropouts, because the wider field of view provided by such optics gives the tracking system additional time to adjust the camera""s aim before the OBT leaves the camera""s field of view. In fact, if a maximum speed for all possible OBTs is known in advance, tracking dropouts can be completely eliminated by using optics with a sufficiently wide field of view.
Unfortunately, using wide-angle lenses to avoid tracking dropouts has two undesirable side effects. First, the resolution of images obtained using even moderately wide wide-angle optics is lower than the resolution of images obtained using narrow-angle lenses. This reduced resolution may result in images that are not suitably sharp for certain applications. For example in security systems, the operator may require a clear, undistorted image of the target at all times.
A second undesirable side effect of wide-angle optics is the distortion that occurs near the top, bottom, and sides of images obtained using wide-angle lenses. This distortion increases as the field of view of the lens gets wider, and is particularly pronounced with fisheye optics.
One prior art system that provides the superior tracking performance of wide-angle optics without sacrificing image quality uses two cameras: a fixed-position, wide-angle, lower-solution camera; and an aimable, narrow-angle camera. In this system, the OBT""s position can always be located using the image data obtained from the wide-angle camera, no matter where or how fast the OBT moves. A tracking system then uses the position information obtained from the wide-angle image to aim the narrow-angle camera at the OBT. A high resolution image of the OBT can be then be acquired using the narrow-angle camera.
With this two-camera prior art arrangement, tracking dropouts can be eliminated or minimized, because even if the OBT moves out of the field of view of the narrow-angle lens, the OBT can be reacquired using the image information captured by the wide-angle image camera. Unfortunately, this two-camera approach is relatively expensive and bulky, because it uses two independent cameras.
A number of systems have been described in the prior art for producing wide angle views as well as narrow angle views. These systems are designed for special effects or for producing multiple image-types, but not in connection with video tracking. For example, one system produces a panorama view and a direct view in the same system. The panorama view remains fixed and the direct view can be changed. A single camera captures both the panoramic view and the direct narrow-angle view on a single CCD image.
There are various other references describing a special-effects optical system. One is Patent Abstracts of Japan Publication No. 09139878A, which describes a lens with a first region having a first focal length designed to focus on a background subject, and a second region having a second focal length designed to focus on a foreground subject. Similarly, Patent Abstracts of Japan Publication No. 3-194502 (A), describes using a first lens with a first focal length and a second lens with a second focal length to simultaneously form a normal and a magnified-view of a single object. Also, Canadian Patent Application 2,147,333 discusses an optical system that includes a fisheye lens that produces a wide-angle image of everything in front of the optical system, and combines it with an image of objects that are behind the camera. The above optical systems are designed for special effects and, in general, are unrelated to video tracking.
A system that employs the advantages of a wide angle view for tracking purposes and the excellent image quality obtainable with narrow angle optics is desirable. Although various optical systems exist for generating multiple views on the same image device, none relates to the field of video tracking or discusses this problem. In the field of video tracking, no good solutions to the conflicting problems of tracking the subject and forming a high quality image in a video tracking system are described.
The present invention relates to a tracking system that provides high quality images and a low tracking dropout rate. The system uses a camera equipped with dual-angle optics to obtain both a wide-angle image and a narrow-angle image of a scene. The narrow-angle image provides a high resolution view of the scene, and the wide-angle image is used for finding and following the OBT. If the OBT moves out of the high resolution area, information obtained using the wide-angle, lower resolution image is used to find the OBT. The aim of the camera is then adjusted so that the OBT will once again appear in the narrow-angle image.
One aspect of the invention relates to a camera that includes an image sensor and a set of optics. When the camera is aimed at a point of a scene, the optics form a narrow-angle image of the point and a central area surrounding the point on a central region of the image sensor, and form a wide angle image of an area surrounding the central area on a peripheral region of the image sensor.
Another aspect of the invention relates to an apparatus for optically tracking a subject. The apparatus includes a camera, optics, a movable base, and a tracking system. The camera has an image sensor with a first region and a second region, and an output. A central portion of the optics produces a narrow-angle view, and a peripheral portion of the optics produces a wide-angle view. When the camera is aimed at a point, the central portion of the optics forms an image of the point and a central area surrounding that point on the first region of the image sensor, and the peripheral portion of the optics forms an image of an area surrounding the central area on the second region of the image sensor. The camera is mounted on the movable base, which is configured to aim the camera in response to drive signals applied to at least one input. The tracking system generates these drive signals based on the output of the image sensor. When a desired subject is imaged by the second region of the image sensor, the drive signals generated by the tracking system cause the movable base to aim the camera at the desired subject.
Another aspect of the invention relates to an apparatus for optically tracking a subject. The apparatus includes an image sensor, an optical system, a movable base, and a tracking system. The image sensor has a first region, a second region, and an output. The optical system, which could include, for example, lenses and/or mirrors, has a first portion with a narrow-angle view and a second portion with a wide-angle view. When the optical system is aimed at a subject, the first portion of the optical system forms an image of the subject on the first region of the image sensor, and the second portion of the optical system forms an image on the second region of the image sensor. The optical system is mounted on the movable base, which aims the optical system in response to drive signals. Based on the output of the image sensor, the tracking system generates these drive signals. When the subject is imaged by the second region of the image sensor, the drive signals generated by the tracking system cause the base to aim the optical system at the subject.
Another aspect of the invention relates to a method of tracking an object in a scene using an aimable wide-angle imaging system and an aimable narrow-angle imaging system. The method operates by obtaining a wide-angle image of the scene using the wide angle imaging system and finding a position of the object in the scene based on this wide-angle image. Then, both the narrow-angle imaging system and the wide-angle imaging system are aimed at the found position.
Another aspect of the invention relates to an apparatus for optically tracking a subject. The apparatus includes a camera with a pan, tilt, and zoom (PTZ) base, and a tracking system. The camera has an output and a zoom lens with a narrow-angle setting and a wide-angle setting, one of which is selected in response to a zoom signal. The switchover between zoom and narrow-angle settings must be as rapid as possible to insure against loss of the OBT. The camera is aimed in response to aiming signals. Based on the camera output, the tracking system tracks a subject when the zoom lens is set to the narrow-angle view. If the tracking system loses track of the subject, the tracking system generates a zoom signal that sets the zoom lens to the wide-angle view and locates the position of the subject based on the camera output. Then, the tracking system generates an aiming signal that aims the camera at the located position of the subject, and generates a zoom signal that returns the zoom lens to the narrow-angle view.
Another aspect of the invention relates to a camera. The camera includes a planar image sensor with an image plane, at least one first lens, and at least one second lens. The first lens(es) has (have) a first optical axis and a first focal length, and the at least one second lens has a second optical axis and a second focal length. The second focal length is longer than the first focal length, and the first and second optical axes are substantially collinear. The first and second lenses are positioned to form respective images on the image sensor.